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ABS TRACT 


Two methods of obtaining sensitivity data were simulated on an 
electronic computer for the purpose of comparing the accuracy of the 
estimates of the parameters of an underlying cumulative normal response 
function. The first method simulated the standard Bruceton procedure 
while the second used a modified binary search routine with a portion of 
the sample in order to obtain maximum likelihood estimates of the input 
parameters for use in a follow-on Bruceton test. 

The results showed both methods to be effective in estimating the 
mean but with slightly more variability in the estimates obtained by the 
second procedure. Both methods underestimated the standard deviation - 
again with more variability in the estimates obtained by the second 
procedure. When the prior parameter estimates were unknown am the 
applicable stimulus level bounded, the second method yielded estimates 
favorably comparable to those expected from the Bruceton procedure with 


suitable prior input estimates. 
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I. INTRODUCTION 


Frequently a statistician is faced with the problem of determining 
the level of a stimulus which critically affects the performance of a 


device. The nature of the 


testing to be discussed is such that once 


Te a 
metpuamemngrercs + — ~— 


some positive level of the stimulus is applied to the device either a 
response or a non-response can be immediately observed and, in either 
case, the device is altered so that a bonafide result cannot be obtained 
from a second test. Tests of this type are known as sensitivity tests. 

One of the many problems besetting those involved in explosives 
research is that of providing measures and specifying rules to provide 
for the safe handling and transportation of explosives. Many different 
types of sensitivity testing apparatus have been developed for laboratory 
use, the most common being those that subject some quantity of explosive 
to the impact load of a falling drop-weight from some controllable height. 
At least as late as October 1965 there remained two important physical 
problems to be solved; namely, that of establishing a measure of stimulus 
not highly apparatus-dependent and then that of translation of these 
results to safe handling rules [1]. These problems are not addressed in 
this paper but should be kept in mind when considering the overall problem. 

In the early 1940's, a technique for obtaining sensitivity data was 
developed and used in explosives research at the Explosives Research 
Laboratory, Bruceton, Pennsylvania which has come to be called synony- 
mously, the Bruceton, Staircase, or "Up and Down" Method. 

The aim of this method of testing is to increase the accuracy with 
which certain critical values of the stimulus may be estimated, notably 


the median (or mean) and standard deviation. The accuracy of the method 








depends in part on the stimulus level at which the first item is tested 
and the interval spacing for subsequent levels of testing [2]. 

When the stimulus levels mentioned above cannot be determined prior 
to testing or when little confidence is placed on the available estimates, 
a preliminary (or search) phase of testing may be desirable to obtain 
maximum likelihood estimates prior to employing the Bruceton Method with 
the remainder of the sample. A procedure to do this is offered as an 
alternative method. 

The comparative accuracies of the two techniques were examined 
through the use of simulation conducted on a high-speed electronic 
computer. All parameters and estimates considered as inputs to the 
simulation were kept within ranges for which the Bruceton Method is 


considered to yield accurate results [2]. 





II, THE MODEL 


Let x be an applied stimulus level (x-[o,~”)) and y = y(x) be the 
associated response (ye do, 1} where "o'' denotes no response and "lL" 
denotes response). At any given stimulus level consider y to be the 
realization of a Bernoulli random variable, Y, with response 
probability 

p(x) = Prob (Y = 1]x) 
The function p(x) is called the response function and is further 


specified as 


p(x) = 0 xe [o,a] 
0< p(x) < l xe(a,b) 
and p(x) = 1 xe [b,~) 


The intervals [0,a], (a,b) and [b,~) are called the zero-response 
region, the mixed-response region, and the one-response region 
respectively. It is assumed that p(x) is a monotonely increasing 
function for stimulus values in the mixed-response region. Thus, 
p(x) can be considered as the cumulative distribution function for a 
random variable X such that 

p(x) = Prob (X <x). [3] 
In this context the random variable X can be interpreted as a thres- 
hold stimulus level, thus 


Prob (Y = 1]x) 


Prob (X < x) p(x) 


Olx) = Prob (X > x) =1- p(x). [3] 


and Prob (Y 
It is assumed the X is distributed Normal (car that is 
2 
p(x) = w(xlu,o) 


2 
where o(x|uU,0 ) represents the cumulative normal distribution with mean 





2 ; 
\1 and variance o . In particular 


Prob (x Su) = pu) = 0.5. [3] 
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Tit. TESTING METHODS 


A. BRUCETON METHOD 
ire Deser hpiian 

Based on intuition or past experiments, the experimenter selects 
a priori estimates of U and o. Call these estimates Hy and OF and let 

The experimenter tests the first item at or near Ly- If there 
is a response the second item is tested at a level d units below Uy» 
otherwise the second item is tested at a level d units above Ly: In 
the same manner, each of the remaining items is tested at a level d 
units above or below the previous test level according as there was not 
or there was a response observed for the previous test. Thus the 
sample is concentrated about the mean and one would expect nearly equal 
numbers of responses and non-responses. In fact, the number of non- 
responses at any level will not differ by more than one from the 
number of responses at the next higher level [2]. 

Let N denote the total number of observations of the less 


frequent event and Ny »MyNys** denote the frequencies of this event 


Ths 
at each level where Ny corresponds to the lowest level and nN the 
highest level at which the less frequent event occurs. 

The final estimates of U and o are based on the first two 


moments of the stimulus levels. Since the intervals are equally 


spaced, these moments can be computed in terms of the sums 


A= in. 
; aL 
i 
and B= % i” n.- 
i 
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Bet (i be the estimate of wu by this method. Then 


i=x'+aG+5) 


where x' represents the lowest level at which the less frequent event 
occurs [2]. The plus sign is used when the analysis is based on non- 
responses, and the minus sign when it is based on responses [2]. 


If (NB-A?)/N° > .3 the sample standard deviation is 


= 029 ) 


Otherwise, a more elaborate calculation must be employed and is 


2 





5 eG al PS 
N 


described in Ref. 2. 
To obtain confidence intervals, estimates of the standard 
deviations of the sample mean and sample standard deviation, say ae 


and S respectively, are given by 


and 


where the factors G and H are dependent on the ratio © and the 
position of the mean relative to the testing levels. Plots of these 
factors are available in Ref. l. 
2. Discussion 

Only rarely is the threshold stimulus Z normally distributed. 
It is usually the case that some scale transformation of Z, say X, is 
made so that X is normally distributed in the vicinity of the mean. 
This transformation is done prior to testing to determine Uy and OF: 
Only after all analysis is completed are the values scaled back to the 


original stimulus measure [2]. 


12 





The size of the sample is critical to the accuracy of the 
estimation. Note that at most only half of the sample is used in the 
analysis so that, for example, if thirty items are tested the maximum 
possible value of N is fifteen. The analysis is based on large sample 
theory which in the case mentioned would be applied to a sample of size 
fifteen [2] [4]. | 

Unless normality of the variate is assured this method does 
not yield accurate results for the small énd large percentage points. 
This is unfortunate since in most applications one would be more 
interested in a small percentage point as a measure of safety and a 
large percentage point as a measure of reliability. At any rate, an 


estimate of a percentage point j is 


A 
j = ut ks 
where k is chosen from tables of the standard normal deviate to give 
the desired percentage [2]. One could then conduct tests in the 


vicinity of this value to refine the estimate. 


B. BRUCETON METHOD PRECEDED BY SEARCH 
1. Description 

In the event that a priori estimates of wu and o are not 
available some economic method of attaining these estimates is desired. 
A method proposed and described below is a modified binary search 
technique. 

Again, the assumption is that the threshold stimulus (or 
some transformation of it) is normally distributed and p(x) can be 
represented by a cumulative normal distribution. 

As noted from the model 


Prob (Y = O|x < a) = 1 
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and 

Prob (Y = 1|x > By = 1. 
The first step in the procedure, then, is to select values for a and b. 
(In the case of complete uncertainty these could be the limiting values 
of the testing apparatus) and commence the binary search starting at 

x = (a + b)/2. 
If p(x) were a step function, repetition of this method would locate 
the step in an interval of any desired length. In general, however, 
the mixed-response region has non-zero width and a non-response would 
merely indicate that the applied stimulus is in the mixed response 
region or below while a response would indicate that it was in the 
mixed response region or above. 


If a test at x, yields a response and a test at x 


1 yields a 


Z 


non-response while xy < Xo ih is scekcain: thateboen xy and X» are in the 
mixed response region. This condition is called a response inversion 
and is the basic indicator for the modified binary search technique. 
The description of the procedure is best followed by referring to 
Figures 1 through 4. 

Sequence S* is a cyclic one indicating that a reduction in step 
Size should be taken. Test levels are selected attempt ing to reproduce 
this sequence. Faiiure to do this results in the basic inversion 
Sequence So. Tests are then made at the end of this sequence to result 
in one of three terminal situations Sy. So > or S3- In the event the 
mixed response region is relatively narrow and near a or b, several 


binary reductions may be necessary to reproduce S* or one of the 


terminal situations. These circumstances are represented by So and 


Sy (7) 
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STARTING SEQUENCE 


Figure l 
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S,, SEQUENCE 


Figure 2 
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S* SEQUENCE 


Figure 3 
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S, SEQUENCE 


Figure 4 
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Maximum likelihood estimates of Ut and o are available for 


and S, and developed as described below [3]. 


Sequences 54> So: 3 


2. Discussion 


It is assumed that all trials are independent. Thus the 





probability of the sequence Sy is 
Eaeb (S,) = Prob (¥,=0, Y¥,=0, Y,=1, Y,=0, ¥,=1, NOX Nak) Xiee ke) 
6 

= pan Prob (Y, - y,|x,) 

where 
Prob (Y. = y.|x,) = ox, ) if ve 1 
=] - p(x.) nie a 0 
and 9 
x. 1 GOD 
p(x.) = Prob (X, < x.) = : e 2 ae dx. 
1 Le a Jon ° 


Maximum likelihood estimates for u and o can then be established 
using standard normal tables for each of the terminal situations. 


These estimates are indicated on Figure 5. 
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IV. SIMULAT.ON 


A. DESCRIPTION 

All simulated experiments were conducted on an IBM 360/67 computer 
using the FORTRAN IV programming language. The basic program " 
attached. The response function p(x) used was cumulative normal with 
uu = 30 and o = 3. 

The sample size was kept at seventy for each experiment to provide 
some assurance that the analytical sample would be suitable for large 
sample analysis. 

The basic test procedure was to draw a random number on the unit 
interval and compare this to F(x), a function of a standard normal 


variate specified as 


F (x) = [1 - erf ) if x < 0 


and 
F(x) = 5 E + erf (=) iia 0 
We 
where FF 2 
erf(v) =| x dite 
At 


(The function subprogram erf is an IBM-supplied subprogram.) If the 
random number was less than or equal to F(x, ) then a response was 
th . 
counted for the i level; otherwise a non-response was counted. 
Six different cases were tested using the straight Bruceton pro- 
cedure (METHOD 1) with two different input estimates of uw and three 
different input estimates of o. Case 1 considered exact estimates; 


ae i, MW, =H and Oo, = 0. Case 2 considered Wy = U-6 and 0, =o. 


~ 
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Cases 3 and 4 considered Uy =H and O, = o/2, 20 respectively while 
Cases 5 and 6 repeated Cases 3 and 4 except Wy = U-6. For each of the 
six cases 1000 experiments were conducted each utilizing a different 
sequence of random numbers. 

The search procedure (METHOD 2) was then incorporated into each 


of the above six cases using the a prior estimates, Ly anid oe Oo 


I’ 
determine estimates for stimulus levels a and b and thereby the size 
of the binary reduction as indicated in Figure 1. The program then 
followed the flow shown in Figures 1 through 4 until either a terminal 
sequence was reached or the search was arbitrarily terminated as dis- 
cussed in subparagraph C below. The Bruceton procedure was then used 
until the sample was exhausted. 

The final case, Case 7, indicated complete lack of knowledge of yu 


and o but considered the upper and lower stimulus level limits of the 


test apparatus to be 100 and O respectively. 


B. MEASURES OF EFFECTIVENESS 
At the completion of all experiments for each case, several 
measures were obtained for comparison. First, average values of the 


parameters were determined to be 


x A 
=r u,/N 
i 
and 
6 =£0,/N 
1 


m a oes le 
where Uy and oO. are the a posteriori estimates of UW and o for the i 


i 


experiment and n, the number of experiments used. Next, as measures of 


variability 





d 2 
E(u, - yw) /n-1 
1 


> 


and 


yr (C, - g)/n-1 
1 


<) 
Q> 
I 


were calculated. In addition, the program listed the maximum and 


minimum estimates of both yw and o. 


C. DISCUSSION 
In Chapter III it was noted that sequences S%, Sip and Sa are 
cyclic. In order to simplify the program it was necessary to 


artificially terminate these situations at some point and calculate 


the input values for the Bruceton test. The estimate of u used was 


uo = (x, + x, )/2 


S 
' where x) and x, are adjacent testing levels and Xy > X) with y, = 0 


and oe 1. The estimate of o used was 

ais (x, 5 x,)/2 
for Cases 1 through 6 and 

ee (x5 i x,)/6 
for Case 7. The former estimate of o was chosen arbitrarily while the 
latter estimate was based on the estimate of the mixed response region 
being 60. While the number of terminations of this type was insignifi- 
cant for the first six search cases, in the final case over 600 experi- 
ments were thus terminated requiring the program to be expanded to 
permit more recycling. The point is that the artificial termination 
does not represent the search procedure. This problem would not arise 
in field experimentation until either the sample was exhausted or the 


step size reduction of stimulus level indicated was too narrow to be 


measured or controlled by the test apparatus. 
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Also in the interest of program simplification those experiments 


for which 





¢ 


Were not used for analyses. This limitation invalidated the measures 


of effectiveness for the Bruceton cases where Oy = 20. 


D. RESULTS 

The results of the simulation are listed in Table I. It is 
questionable that the measures listed under Method 1 are valid for 
Cases 4 and 6 in that only -381 and .393 of the possible experiments 
were used. These two cases and Case 4 under Method 2 (where .661 of 
the possible experiments were used) are the only ones for which 
6 >. 

In general the extreme estimates are more widely separated and the 
variability of ¢ is greater in Method 2. 

Estimates of uy range from 27.8823 to 31.7647 for Method 1 and 
Zeee9s/ to 31.91 for Method 2. 

Estimates of o range from .8741 to 6.5027 for Method 1 and .3498 
to 9.8328 for Method 2. 

The lowest average Q, 29.9113, was obtained under Method 1, Case 5, 
while the highest average Ch, 30.1175, was obtained under Method 2, 
Case 3. 

The lowest average C; 2.3748, was obtained under Method 2, Case 5, 
while the highest average S, 2.9474, was obtained under Method l, 

Case 5. (Case 6 is not counted under Method 1 nor is Case 4 under 


both methods.) 
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TABLE OF EXPERIMENTAL RESULTS 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 
1. Estimation of the Mean 
Both methods estimate the mean effectively. 
2. Estimation of the Standard Deviation 
Both methods tend to under-estimate the standard deviation with 
no predictable bias and are therefore unsuitable for use in safety or 
reliability statements. This conclusion agrees with the findings of 
Hampton [4] as it pertains to the Bruceton Method. 


3. Extension of the Search Phase for the Starting Sequence 


Termination of the search phase with sequence S, in the starting 


1 
sequence (see Figure 1) may yield estimates of o greater than twice 
the actual value. To avoid this it is advisable to extend the search 
phase as described in Ref. 3. 
4. Use of Search Technique 
The search procedure should be used in those cases where there 
is not independent evidence that the estimate of o is within the range 


for which the Bruceton Method is recommended (i.e., o/2 < o. < 20). 


I 


B. RECOMMENDATIONS 

Further testing of Method 2 is recommended under the circumstances 
listed below. 

1. Reduction of Sample Size 

It would be of in*erest to reduce the sample size to the point 

where the effective sample is small, say 15, and compare the Bruceton 
procedure with the search procedure using the entire sample for the 
search. 


26 


2. Random Selection of Response Function Parameters 


A more valid test of both methods would be achieved by 
randomally selecting values of Wy and o over some range and using 


the on-line computer facility to conduct the simulation. 
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